Keeping your SSH connections alive with autossh

With autossh, you can monitor your SSH connections and restart them if they stop sending traffic or SSH exits abnormally. This makes autossh perfect for keeping secure port forwarding available.

Some readers will be familiar with the TCPKeepAlive, ServerAliveInterval, and ServerAliveCountMax options to SSH itself. TCPKeepAlive causes TCP keepalive messages to be sent to the server, allowing SSH to detect if it can no longer contact the server. ServerAliveCountMax and ServerAliveInterval cause the SSH client to send traffic through the encrypted link to the server, and can be used both the avoid a connection being closed due to inactivity and to have the SSH client exit if traffic cannot be returned from the server for a specified amount of time. The default action if TCPKeepAlive messages are ignored or ServerAliveCountMax messages are not returned in time is for the SSH client to exit.

While you can disable TCPKeepAlive and ServerAliveInterval if you are using SSH for port forwarding, you will then not get messages informing you that the port forwarding is inactive due to the network or server being unresponsive. Using autossh to control your SSH client means you can turn on TCPKeepAlive and ServerAliveInterval so transmission failures are detected and have autossh attempt to reestablish your SSH connections automatically when this happens.

Packages for autossh are in the standard repositories for Ubuntu Hardy, Fedora 8, and as a openSUSE 10.3 1-Click Install. In this article I’ll use autossh version 1.3-4 on some Fedora 8 machines.

Options

The command-line options to autossh are extremely spartan. The utility uses mostly environment variables instead. The intention is that there should be very little change to your command lines, ultimately leading to the ability to simply replace the ssh command name with autossh. If you have not set any special environment variables, the only extra option that you have to use for autossh is -M port[:echo_port]. You can avoid having to use the -M option as well by setting the AUTOSSH_PORT environment variable. The -M and AUTOSSH_PORT options tell autossh how it should send heartbeat messages to the server.

autossh supports two different heartbeat designs; one uses port forwarding only and one needs an echo server at the remote end. The port forwarding only option sets up a port forwarding from the client to the server and another one from the server back to the client. When autossh sends a packet on the port forwarding to the server, the second port forwarding rule sends it back to the client again. This means that autossh can operate without any modifications to the server at all. To use the port forwarding only solution using ports 20000 and 20001, pass -M 20000 to autossh or set the AUTOSSH_PORT environment variable to 20000. If you want to use the echo server method, use -M 20000:7, where seven is the normal port number that an echo server runs on.

Other than the AUTOSSH_PORT environment variable, other settings you can change include the initial gatetime, what level of and where logging occurs at, and how frequently to poll the server to check whether the SSH connection is alive. The gatetime is how many seconds must have passed before autossh considers the SSH connection to have been initially successful. For port forwarding SSH connections and initial testing, I would recommend setting AUTOSSH_GATETIME=0. The downside of the default value of 30 is that if SSH cannot connect the first time, it will exit, and autossh might assume that the connection is impossible and not attempt to restart the connection. A gatetime of zero means that autossh will always try to restart all SSH connections. For logging, AUTOSSH_DEBUG will send messages to stderr as well as syslog. AUTOSSH_LOGFILE and AUTOSSH_LOGLEVEL can be used to obtain more logging verbosity and to redirect logs to a nominated file instead of syslog. AUTOSSH_POLL sets the number of seconds between heartbeat messages being sent.

Because you are likely to be using it to keep an SSH connection up, autossh will try to obtain any needed authentication information from an ssh-agent. If there is no agent running then autossh will start one for the session. It is best to allow the connections monitored by autossh to be restarted without human intervention, as this is likely the main reason that you are using autossh in the first place.

Running

In the example below I have set the two environment variables that I recommend you always use, as well as the AUTOSSH_DEBUG variable to have autossh show in detail want is happening. Because the port is set as an environment variable I can simply invoke autossh instead of ssh as the only change to the command itself.

[root@client1 .ssh]# set|grep AUTOSSH
AUTOSSH_DEBUG=1
AUTOSSH_GATETIME=0
AUTOSSH_PORT=20000
[root@client1 .ssh]# autossh server
autossh[21170]: checking for grace period, tries = 0
autossh[21170]: starting ssh (count 1)
autossh[21171]: execing /usr/bin/ssh
autossh[21170]: ssh child pid is 21171
autossh[21170]: check on child 21171
autossh[21170]: set alarm for 600 secs
Last login: Tue Apr 29 19:12:08 2008 from client1
[root@server ~]# exit
logout

Connection to server closed.
autossh[21170]: check on child 21171
autossh[21170]: ssh exited with status 0; autossh exiting
autossh[21170]: expired child, returning 2
[root@client1 .ssh]# 

If you wish to signal the ssh process to exit, you will find it in the output of ps aux including some extra port forwarding specifications. For a AUTOSSH_PORT=20000, something like the command shown below will appear in your ps output:

# ps aux
....
root     22337  0.1  2.8  61384 14640 pts/0    S+   19:22   0:00 ...
  ... /usr/bin/ssh -L 20000:127.0.0.1:20000 -R 20000:127.0.0.1:20001 server

If you kill the ssh command with a signal 15 (SIGTERM), autossh will restart your connection as shown below. Sending a signal 9 (SIGKILL) to the ssh process will cause autossh to exit as well instead of restarting the SSH connection.

[root@client1 .ssh]# autossh server
autossh[22323]: checking for grace period, tries = 0
autossh[22323]: starting ssh (count 1)
autossh[22323]: ssh child pid is 22326
autossh[22323]: check on child 22326
autossh[22323]: set alarm for 600 secs
autossh[22326]: execing /usr/bin/ssh
Last login: Tue Apr 29 19:17:23 2008 from client1
[root@server ~]# 
Killed by signal 15.
autossh[22323]: check on child 22326
autossh[22323]: ssh exited with error status 255; restarting ssh
autossh[22323]: expired child, returning 1
autossh[22323]: checking for grace period, tries = 1
autossh[22323]: starting ssh (count 2)
autossh[22323]: ssh child pid is 22337
autossh[22323]: check on child 22337
autossh[22323]: set alarm for 592 secs
autossh[22337]: execing /usr/bin/ssh
Last login: Tue Apr 29 19:18:06 2008 from client1
[root@server ~]# 

Wrap up

When you have some port forwardings that you want to always be up, autossh is a great tool. The command-line interface is similar by default to normal SSH invocation, which allows you to quickly configure autossh to monitor an SSH connection. The heartbeat mode that uses port forwarding only lets you quickly start using autossh without any modifications to the server.